Lecture 19: Spectral Clustering

We are given n data points x1, . . . , xn and some way to compute similarities between them, call si,j the similarity between the n points. Assume that the similarity function is symmetric, so si,j = sj,i. For the purposes of this lecture let us just consider the simplified clustering problem where we would like to split the dataset into two clusters. We would like to find a subset S ⊂ [n] of size roughly n/2 such that the similarity between points in S (and S) is high, while the similarity between the clusters is low. We do not assume that we have any object features, just the similarities si,j . Using these, we construct a similarity graph G = (V,E) where the vertices are the n objects, so |V | = n and the edges are weighted with si,j . In graph-theoretic terminology, we want to partition the graph so that the edge between the partitions have low weight while the edges within each side of the partition have high weight. Let us form the adjacency matrix of this graph, which we will always call A ∈ Rn×n. This matrix has Ai,j = si,j and is unspecified on the diagonal (don’t worry we won’t really use it at all). Define the degree matrix D ∈ Rn×n which is a diagonal matrix with Di,i = ∑n j=1 si,j , the (weighted) degree of vertex i in the graph.